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Abstract Preschoolers with elevated anxiety symptoms 
are at high risk not only of developing more severe mental 
health disorders in later life but are also apt to respond 
more poorly to intervention if they present with comorbid 
disruptive behavior. Because early signs of anxiety disor- 
ders may not be recognized as such in preschool settings, 
many children selected for Tier 2 interventions that target 
externalizing problem behaviors may also have co-occur- 
ring anxiety symptoms and disorders. The First Step to 
Success intervention has recently been adapted for 
preschoolers with externalizing behaviors and was found to 
be efficacious in a randomized controlled trial. The current 
report examines effects of the First Step intervention on a 
subsample of 38 preschoolers with comorbid anxiety 
symptoms. Compared to usual-care controls, preschoolers 
who were assigned to the First Step intervention demon- 
strated medium-to-large effects in reducing externalizing 
behavior and improving social functioning outcomes, but 
had small effects for reductions in internalizing behaviors. 
Implications for intervening with preschoolers at risk of 
comorbid disruptive and anxiety behaviors are discussed. 
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Introduction 


Anxiety disorders are among the top five health burdens in 
terms of their disabling impact within US populations (U.S. 
Burden of Disease Collaborators, 2013). Unfortunately, 
anxiety disorders are also the least likely of the major 
pediatric mental health disorders to be treated across the 
main mental health service delivery systems including 
mental health, general medicine, social services, juvenile 
justice, and school settings (Merikangas et al., 2011). The 
most efficacious evidence-based interventions for anxiety 
disorders for children and adolescents are generally con- 
sidered to include a combination of pharmacotherapy and 
cognitive—behavioral treatments (Franklin et al., 2011; 
Piacentini et al., 2014). However, there is limited efficacy 
research on early interventions targeting anxiety disorders 
among children. The dearth of trained therapists and 
reluctance of families to medicate their young children are 
formidable barriers to accessing evidence-based practices 
(Ryan, 2014). As Luby (2013) notes, “early interventions 
during a time of greater developmental change holds pro- 
mise to be an area where larger and more enduring treat- 
ment effects are possible” (p. 355). 

Preschoolers with disruptive behavior, who do not 
readily respond to universal classroom management 
strategies in Head Start or other early childhood class- 
rooms, may often be selected for more intensive second- 
tier interventions. Although this disruptive behavior may 
seem obvious to teachers and even parents, accompanying 
comorbid anxiety disorders may go unrecognized and may 
not be specifically targeted by these broad-based, generic 
interventions. Such interventions primarily involve 
behavioral approaches that target a wide range of disrup- 
tive behavior disorders (Forness, 2011). The differential 
impact, however, of such behavioral intervention 
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approaches on preschoolers who are also at risk of het- 
erotypic comorbid psychiatric disorders such as disruptive 
behavior and anxiety disorders has not been well estab- 
lished (Luby, 2013). This is an important issue given that 
anxiety disorders not only have high comorbidity with 
disruptive behavior disorders (Franz et al., 2013), but such 
comorbidity also portends poorer treatment outcomes 
(Garcia et al., 2010; Hudson et al., 2015; Rapee et al., 
2013). Furthermore, even after accounting for comorbid 
disorders, parents of children with anxiety disorders are 3.5 
times more likely to report a negative impact on family 
functioning relative to parents of children without anxiety 
disorders (Towe-Goodman, Franz, Copeland, Angold, & 
Egger, 2014). 

First Step to Success is a broadband, Tier 2 early 
intervention originally targeted to K-third grade students 
with externalizing behavior problems involving disruption, 
aggression, and oppositional—defiant forms of behavior 
(Walker et al., 1998). Based upon a social learning theo- 
retical framework, First Step is an early intervention that 
consists of three modular components: (a) universal 
screening to identify qualified candidates for the program; 
(b) a school intervention component focused on improving 
teacher- and peer-related adjustment along with overall 
school success; and (c) a parent-training component, called 
homeBase, in which parents are taught to teach their child 
school success skills at home which are then noted, 
prompted, and reinforced by the child’s teacher(s). The 
First Step program is designed to assist students with 
externalizing behavior problems in getting off to a good 
school start and has been the subject of a continuous series 
of studies since it was developed (see Walker et al., 2014, 
for a review and update of the evidence base on First Step). 
Recently adapted for use with preschoolers, First Step was 
found to be efficacious in a large randomized controlled 
trial (Feil et al., 2014) that forms the basis for the current 
study. 

First Step has thus focused primarily on, and has been 
quite effective with, disruptive behavior. However, it is 
not clear to what extent its effectiveness might be 
impacted by the presence of comorbid anxiety disorders as 
noted above. In a recent comprehensive review on treat- 
ment of preschoolers with anxiety disorders, Luby (2013) 
notes only five relatively small studies had been conducted 
that targeted anxiety problems at the time of the review. 
The anxiety treatments were primarily developmental 
adaptations of cognitive—behavioral interventions and tend 
to emphasize parent psychoeducation and active teaching 
and coaching during parent-child interactions. There is 
reason to think, however, that there are certain aspects of 
First Step that might at least indirectly address anxiety 
problems. First Step, for example, includes intervention 
components that have been shown to ameliorate anxiety 
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symptoms such as extensive social skills training (Rapee 
et al., 2013; Spence, 2003) and parent coaching and 
training (Bernstein, Bernat, Victor, & Layne, 2008; Rapee, 
Kennedy, Ingram, Edwards, & Sweeney, 2005; Wood, 
Piacentini, Southam-Gerow, & Sigman, 2006). As such, 
we hypothesized that clinically meaningful intervention 
effects might be achieved for internalizing symptoms 
because social isolation (Mathews et al., 2015) and 
behavioral inhibition (Rapee, Kennedy, Ingram, Edwards, 
& Sweeney, 2010) are considered critical aspects of anx- 
iety disorders and their treatment. Students with severely 
disruptive or aggressive behavior problems can often be 
isolated due to the aversive nature of their behavioral 
characteristics (Walker & Gresham, 2014). A unique 
feature of First Step involves use of a combined individ- 
ualized and group-dependent contingency wherein the 
classroom focus is on providing not only a system of 
reinforcement for the target child but also an opportunity 
for the target child to earn a brief rewarding activity for 
classmates if he or she meets a daily performance crite- 
rion, thereby increasing social acceptance and decreasing 
behavioral isolation by peers. Thus, although First Step 
does focus substantially on disruptive behavior, it includes 
elements that might nonetheless prove helpful for young 
children with comorbid anxiety disorders. Although these 
First Step components do not necessarily focus on anxiety 
symptoms per se, they are meant to increase general social 
and adaptive skills that could help to ameliorate forms of 
anxiety-driven behavior. 

The purpose of the current study, therefore, was to 
extend the efficacy trial findings from Feil et al. (2014) by 
conducting secondary analyses, using a subsample of par- 
ticipants to determine the effects of the First Step inter- 
vention among preschoolers with comorbid disruptive 
behavior and anxiety problems. Our research questions 
involved whether a subsample of preschoolers with co- 
occurring disruptive behavior problems and anxiety 
symptoms who received the First Step intervention 
demonstrated significantly greater gains compared to con- 
trol participants with respect to (a) externalizing behavior, 
(b) social functioning, and (c) internalizing behavior. As 
noted previously, most Tier 2 interventions do not address 
such heterotypic comorbidity but nonetheless will inevi- 
tably include children at risk of unrecognized comorbid 
psychiatric disorders that may negatively impact outcomes 
of generic behavioral approaches. Because such comor- 
bidity portends poorer treatment outcomes, we wanted to 
examine whether effect sizes on externalizing behavior and 
social functioning in the comorbid subsample would be 
comparable to the medium-to-large effects found in our 
original trial. Furthermore, we wanted to examine to what 
extent, if any, internalizing symptoms would be improved 
as well. 


School Mental Health 


Method 
Participants 


Participants for this study included 38 of 126 student, 
parent, and teacher triads recruited across three cohorts 
between 2009 and 2012 for participation in a randomized 
controlled efficacy trial of the preschool adaptation of First 
Step to Success (Feil et al., 2014). Project staff recruited 
participants from Head Start and preschool programs in 
Oregon, Kentucky, and Indiana. Randomization occurred 
at the classroom level; however, only one student per 
classroom participated in the study. Specifically, the stu- 
dent in each classroom who exhibited the most severe 
externalizing behavior based on teacher report was ran- 
domized either to the First Step intervention condition or 
usual-care control. 


Procedures 


Project staff obtained IRB approval for the study prior to 
recruiting and consenting participating teachers. Consented 
teachers identified children in their classroom who exhib- 
ited elevated externalizing behavior using an adapted ver- 
sion of the Early Screening Project (ESP; Feil, Walker, & 
Severson, 1995). The ESP is a multiple-gated screening 
procedure. Teachers, at screening stage 1, identified and 
rank-ordered five children in their classroom who most 
closely matched a description of externalizing behavior. 
Teachers then completed three stage 2 rating scales from 
the ESP for each child: the adaptive behavior index (ABI), 
maladaptive behavior index (MBI), and aggressive 
behavior scale (ABS). Project staff rank-ordered the five 
students within each classroom based on stage 2 rating 
scale scores to identify the highest ranked child. When 
unable to recruit the highest ranked child, project staff 
recruited the next highest ranked child in the classroom 
until they obtained parent consent for one child in each 
classroom or the parents of all eligible children in a 
classroom declined participation. 

For the efficacy trial, 61 child—parent-teacher triads 
were randomized to the usual-care control condition and 65 
triads were randomized to the intervention condition. 
Children participating in the efficacy trial were primarily 
male (65%), African-American (31%), or Caucasian 
(44%), and had a mean age of 4 years (SD = .4). Partici- 
pating teachers were African-American (18%) or Cau- 
casian (72%) and nearly all were female (99%). Teacher 
education levels varied with 22% reported holding a high 
school diploma, 33% an associate degree, 23% a bachelor’s 
degree, and 22% a master’s degree or higher. Teachers 
reported teaching on average for 14 years (SD = 9.2). 


Parents and teachers in triads randomized to the inter- 
vention condition received ongoing, one-on-one consulta- 
tion with a behavioral coach. Teachers received support 
implementing the program and parents met weekly with the 
coach for 6-8 weeks with sessions focusing primarily on 
the promotion of school success skills. Intervention 
teachers also received training in the preschool version of 
First Step (described below) as well as in classroom 
management and positive behavior support techniques 
(Golly, 2006; Sprague & Golly, 2013). Teachers in triads 
who were randomized to the usual-care control condition 
received only a brief training session in classroom man- 
agement strategies and positive behavior strategies, 
whereas parents in usual-care triads did not receive any 
support or training from project staff. 

Behavioral coaches held a bachelor’s degree or higher 
and completed a 2-day training session during which they 
received intensive training on First Step implementation. 
Coaches met weekly in person or via phone with lead 
implementers to discuss and troubleshoot cases. Frequent 
fidelity checks were conducted to ensure program adher- 
ence and implementation quality. 

Project staff collected baseline data prior to random- 
ization and implementation training; approximately 
4 months later upon completion of the intervention, they 
collected posttest data. Intervention and usual-care condi- 
tions did not differ on the mean number of days between 
baseline and postintervention (¢[122] = .87, p = .386). 
Postintervention packets were collected an average of 
128 days (SD = 28.6) after baseline for intervention triads; 
for usual-care triads, packets were collected an average of 
133 days (SD = 28.1) after baseline. For further details on 
procedures for the efficacy trial, see Feil et al. (2014). 


Elevated Anxiety Sample 


To identify children at risk of anxiety disorders for the 
current report, we utilized parent report on the Early 
Childhood Inventory-4 (ECI-4; Gadow & Sprafkin, 2000). 
The ECI-4 is a screening tool for childhood behavioral, 
affective, and cognitive disorders and is based on the 
Diagnostic and Statistical Manual of Mental Disorders, 
fourth edition (American Psychological Association, 
1994). For anxiety disorders, the ECI-4 includes DSM 
screening criteria for separation anxiety disorder, general- 
ized anxiety disorder (GAD), social phobia, specific pho- 
bia, obsessive-compulsive disorder (OCD), and 
posttraumatic stress disorder (PTSD). ECI-4 items are 
scored on a four-point frequency scale (never, sometimes, 
often, and very often). To meet categorical symptom cri- 
teria, items for each disorder are dichotomized (i.e., never 
and sometimes = 0, often and very often = 1) and scored 
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in accordance with DSM symptom cutoffs. As a screening 
tool, the ECI-4 assesses symptomatology but does not 
address onset, duration, functioning, or exclusionary cri- 
teria for a disorder. Thus, children who meet criteria for a 
disorder on the ECI-4 should be considered as being “at 
risk” rather than meeting criteria for a formal diagnosis. 
The ECI-4 symptom categories have demonstrated mod- 
erate to high internal consistency (Gadow, Sprafkin, & 
Nolan, 2001; Cronbach « = .63—.95) and moderate to high 
3-month temporal stability (Gadow & Sprafkin, 1997; test— 
retest r = .43-.77). 

Utilizing ECI-4 symptom criteria for separation anxiety 
disorder, GAD, social phobia, OCD, and PTSD, 38 of 126 
children (30%) met criteria for being at risk of developing 
an anxiety disorder (separation anxiety, n = 7; GAD, 
n = 6; social phobia, n = 4; OCD, n = 14; with an addi- 
tional seven children meeting criteria for more than one 
anxiety disorder simultaneously). Based on ECI-4 criteria 
for disruptive disorders, six of 38 children in the sample 
met criteria for ADHD, two met criteria for conduct dis- 
order, another two children met criteria for oppositional 
defiant disorder, and 18 children met criteria for more than 
one disruptive disorder. The remaining 10 children did not 
meet criteria for a disruptive disorder based on parent-re- 
ported ECI-4 data. Although we did not necessarily set out 
to require equal numbers in each condition, the ECI-4 
criteria for anxiety disorders nonetheless resulted in 38 
children being evenly divided between intervention and 
usual-care conditions (1.e., 19 children per condition). 

We compared the sample at risk of anxiety disorders 
identified for this report (n = 38) to the remaining children 
who were not so identified from the sample (n = 88) on 
baseline outcome measures. As reported in Table 1, the 
comorbid and externalizing-only children differed signifi- 
cantly on teacher- and parent-reported internalizing 
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behavior. Children in the sample at risk of anxiety disor- 
ders had higher parent-reported externalizing behavior and 
higher teacher-reported adaptive behavior as compared to 
the externalizing-only children from the original sample. 


The First Step to Success Intervention for Preschool 
Settings 


The First Step to Success intervention is a manualized 
intervention for elementary school students (Walker et al., 
1998). The program includes a classroom-based compo- 
nent (CLASS) and a home-based component (homeBase). 
First Step has been recently adapted for preschool or Head 
Start classroom use and is implemented as part of the 
teacher’s regular classroom routine. The program and its 
adaptations are described below. 


Classroom-Based Component (CLASS) 


The classroom component has three phases completed 
across 30 program days: (a) coach phase, (b) teacher phase, 
and (c) maintenance. The classroom component reinforces 
adaptive behavior patterns intended to enhance school 
success and improve peer relations. During the coach phase 
(program days 1-10), the behavioral coach coordinates 
implementation of the program, working with and coordi- 
nating specified roles of the target child, parent(s), tea- 
cher(s), and classmates. The coach explains the 
intervention to each social agent role-plays with the target 
child, and implements the program for the first 10 days. 
During implementation, the coach provides feedback to the 
target child using a green and red card to signal appropriate 
and inappropriate behavior, respectively. The behavioral 
coach monitors the child’s behavior and, at predetermined 
intervals, awards a point depending on the color of the 


Table 1 Baseline means and 
standard deviations for outcome 
measures by comorbid and 
externalizing-only samples 


M (SD) 


Internalizing behavior 


SSiS-INT-T 5.9 (3.6) 
SSiS-INT-P 8.9 (4.7) 
Externalizing behavior 
SSiS-EXT-T 18.2 (6.1) 
SSiS-EXT-P 20.1 (7.1) 
ESP-MBI-T 28.6 (6.8) 
ESP-ABS-T 20.1 (6.8) 
Social functioning 
ESP-ABI-T 24.1 (4.3) 
SSiS-SS-T 774 (10.4) 
SSiS-SS-P 89.9 (11.1) 


Comorbid sample (n = 38) 


Externalizing-only sample (n = 88) Test statistic 


M (SD) t p value 
4.6 (2.9) —2.16 .033 
5.7 (3.3) —3.80 <.001 

18.0 (5.9) —.20 841 

15.0 (6.4) —3.97 <.001 

29.6 (6.6) .719 431 

20.3 (6.2) 23 .820 

22.1 (4.9) —2.11 .037 

76.8 (11.7) —.28 779 

93.5 (14.4) 1.33 186 
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card. If the child exhibits appropriate behavior (e.g., fol- 
lowing classroom rules, cooperating, sharing, sitting qui- 
etly and attentively during circle time) when the point 
award interval occurs, he or she earns a point. If the daily 
performance criterion of 80% or more available points is 
met, the target child and his or her classmates receive a 
brief rewarding activity selected by the child in collabo- 
ration with the teacher. Parents also arrange for positive 
reinforcement in the home setting each day the criterion is 
met in the classroom by providing an individual activity or 
reward when the child returns home. If the daily criterion is 
not met, that program day recycles (i.e., is repeated until 
the reward criterion is met). 

The teacher takes over implementation under close 
supervision and support from the coach during the teacher 
phase (program days 10-20) of the program. During this 
phase, implementation procedures are identical to those 
described for the coach phase with the duration of the game 
gradually being extended to the entire day. During the 
maintenance phase of the program (program days 21-30), the 
child’s dependence on the structured program features is 
reduced. Praise and expressions of approval from the teacher 
replace earned points. Daily feedback is reduced and rewards 
that are given are contingent upon exemplary performance. 

Younger children in preschool settings require addi- 
tional practice to understand and master the behavioral 
skills and expectations of the program. In turn, the coach 
role-plays with the child before each implementation ses- 
sion. In comparison with the elementary version of the 
program, the coach also provides more supervision and 
problem-solves more frequently. For example, if the 
child’s behavior is inappropriate and he or she does not 
respond to feedback, the coach will determine whether the 
child understands the expectations and, if not, will role- 
play the expected behavior with the child and encourage 
the child to comply. 


Home-Based Component (HomeBase) 


During the home component, the behavioral coach meets 
weekly, for a 6—8-week period, with the target child’s 
parents or caregivers. Each weekly session focuses on one 
skill such as communication and sharing, cooperation, limit 
setting, problem-solving, friendship making, and self-con- 
fidence. The coach provides support to the family, helps 
troubleshoot problems and issues that arise during and 
following completion of the program, and _ facilitates 
communication between the home and school. To facilitate 
implementation with younger children, parent meetings are 
conducted with the parent and child so that, if needed, the 
coach can model positive interactions with the child during 
completion of weekly homeBase activities. As well, initi- 
ation of the homeBase component begins after day three 


rather than day ten. For more detailed descriptions of the 
First Step program and adaptation for preschool settings, 
see Feil et al. (2014) and Frey et al. (2013). 


Outcome Measures 


Social Skills Improvement System Rating Scales (SSiS). 
For this report, we examined the teacher-reported and 
parent-reported internalizing behavior subscale, external- 
izing behavior subscale, and social skills scale from the 
SSiS rating scales (Gresham & Elliott, 2008). Items are 
reported on a 4-point frequency scale (Never, Seldom, 
Often, Almost Always). The 7-item teacher-reported ver- 
sion of the internalizing subscale (SSiS-INT-T; « = .73) 
and the 10-item parent-reported version of the internalizing 
subscale (SSiS-INT-P; « = .70) assess internalizing 
behaviors such as anxiety, loneliness, sadness, and low 
self-esteem. The 12-item teacher-reported version of the 
externalizing subscale (SSiS-EXT-T; « = .86) and the 
12-item parent-reported version of the externalizing sub- 
scale (SSiS-EXT-P; « = .90) assess disruptive behaviors 
such as fights with others, disobeys rules or requests, and 
bullies others. For SSiS subscales, standard scores are not 
available; thus, raw scores are reported. Higher raw scores 
on the internalizing and externalizing subscales indicate 
higher levels of symptomatology. The social skills scale 
assesses behaviors that facilitate positive interactions and 
minimize negative interactions (Gresham & Elliott, 2008). 
The 30-item teacher-reported (SSiS-SS-T; « = .93) and 
46-item parent-reported (SSiS-SS-P; « = .95) versions of 
the social skills scale assess common social behaviors such 
as communication, cooperation, assertion, responsibility, 
empathy, engagement, and self-control. For the social skills 
scale, standard scores were computed using gender-specific 
normative data from the SSiS manual. 


Process Measures 


Project staff collected implementation fidelity data, mea- 
sures of teacher and coach alliance, estimates of child and 
parent compliance, measures of parent fidelity and dosage, 
and satisfaction data for triads randomized to the inter- 
vention condition. 


Implementation Fidelity Checklist (IFC) 


The IFC assesses adherence to program guidelines and 
implementation quality for the CLASS component. The 
IFC was adapted to preschool settings from Walker et al. 
(2009). Observers conducted three observations for each 
case: one during the coach phase and two during the tea- 
cher phase. For 20% of the IFC observations, we collected 
interrater reliability data. Interrater reliability was 
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acceptable (ICC[3,1] = .82). The IFC assesses implemen- 
tation of 16 core PFS program components (« = .89). For 
each component, adherence ratings are scored dichoto- 
mously (i.e., yes or no) and quality rating are assessed on a 
5-point scale (0 = very poor, .25 = poor, .50 = okay, 
.15. = good, 1.0 = excellent). Adherence scores represent 
the proportion of observed components implemented cor- 
rectly. Quality scores represent the mean item rating across 
core components. Mean ratings across implementers were 
calculated to examine overall classroom quality. 


Classroom Monitoring Form (CMF) 


The coach and teacher use the CMF daily to log the target 
child’s compliance (Walker et al., 2009). The CMF facil- 
itates tracking of (a) points earned, (b) whether the daily 
criterion was met, and (c) whether a recycle day was 
necessary. CLASS dose is the proportion of 30 program 
days completed. Child compliance is the proportion of 
program days successfully completed to total program days 
(Sumi et al., 2012; Walker et al., 2009). Scores for both 
measures range from 0 to 1. As an example, if a child 
completes all program days without recycling, he or she 
would have a compliance score of 1.0 (1.e., 30/30); if a 
child successfully completes all 30 program days but has 
nine recycles days, he or she would have a compliance 
score of .77 (i.e., 30/39). 


homeBase Monitoring Form (HMF) 


The coach uses the HMF weekly to log the sessions 
completed, parent fidelity, and completion of homework 
assignments (Walker et al., 2009). HMF data are used to 
compute homeBase dosage, parent fidelity, and parent 
compliance. We calculated homeBase dosage as the pro- 
portion of sessions delivered. Dosage scores range from 0 
(i.e., no sessions delivered) to 1 (1.e., all sessions deliv- 
ered). For each session, the coach rates parent fidelity on a 
3-point scale: high (1), medium (.5), and low (0). Parents 
who implement all procedures effectively are rated high; 
parents who exhibit moderate skill and enthusiasm are 
rated medium; and parents demonstrating limited skill, 
interest, and cooperation are rated Jow. Parent fidelity is the 
mean fidelity score across sessions. Higher scores (ranging 
from O to 1) indicate higher parent fidelity. Parent com- 
pliance is the proportion of homework assignments com- 
pleted (range 0 to 1). 


Alliance Survey 
Program implementers completed a 10-item alliance mea- 


sure to assess their collaboration of program implementa- 
tion (Walker et al., 2009). The teacher and coach report on 
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one another’s approachability, shared goals, communica- 
tion skills, willingness to collaborate, and overall effec- 
tiveness. Items range from never to always on a 5-point 
scale. For each informant, we computed the mean rating 
across the 10 items. Scores range from 0 to 5 and higher 
scores indicate higher mean alliance ratings. Coefficient 
alpha was excellent for both the coach version (a = .94) 
and teacher version (« = .95). 


Satisfaction Survey 


Teachers and parents completed a satisfaction survey 
during the postintervention assessment (Sumi et al., 2012; 
Walker et al., 2009). The 13-item teacher survey 
(a = .91) and 12-item parent survey (« = .94) are scored 
on a 5-point scale ranging from strongly disagree to 
strongly agree. Teacher-reported items assess training and 
support received, program usability, and program effec- 
tiveness. Parent-reported items assess program usability 
and effectiveness within the home setting. For each 
informant, we calculated a mean satisfaction rating across 
all scale items. Higher scores indicate higher levels of 
satisfaction. 


Statistical Analysis 


We specified a linear regression model for each outcome 
using Mplus 7 (Muthén & Muthén, 1998-2012). Regres- 
sion models included a covariate (i.e., the baseline value of 
the outcome) and a dichotomous predictor for intervention 
condition (1 = PFS intervention, 0 = usual-care control). 
To account for missing data, we utilized the full informa- 
tion maximum likelihood estimator that makes use of all 
available data to calculate unbiased parameter estimates 
and standard errors (Schafer & Graham, 2002). 

As a measure of intervention effect size, we report 
Hedges’ g, the difference between the mean of the outcome 
measure of each group divided by the pooled within-group 
standard deviation (What Works Clearinghouse, 2011). An 
effect size of .2 is considered small; effect sizes of .5 and .8 
are considered medium and large, respectively. 

As a measure of practical significance, we also report 
the What Works Clearinghouse improvement index (What 
Works Clearinghouse, 2011). The improvement index 
indicates the anticipated change in the percentile rank of an 
average child from the control condition (i.e., 50th per- 
centile) if the child had been randomized to the interven- 
tion condition. 

Given the small sample size (n = 38), this ancillary 
study is considered to be underpowered for detecting 
clinically meaningful effects. For example, for tests of 
group differences at posttest for continuous measures with 
a sample size of 38 the power was < .30 to detect g = .35. 
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Thus, effect size estimates were used to evaluate inter- 
vention effects with g > .35 as the benchmark for assessing 
clinically meaningful results. 


Results 
Preliminary Analyses 


As reported in Table 2, there were no statistically signifi- 
cant differences between intervention and control condi- 
tions on measures of child demographic characteristics and 
screening measures. We also examined baseline equiva- 
lence of outcome measures and parent and teacher demo- 
graphics for the two conditions. For all nine outcome 
measures examined in this report, mean baseline scores 
were equivalent for the two conditions. With respect to 
parent demographics, the two conditions were equivalent 
on all measures, including percent biological or adoptive 
mothers as informants (7 [1, N = 38] = 2.07, p = .150), 
percent in two-parent households (77 [1, VN = 38] = 1.31, 
p = .252), percent Caucasian (x7 {1, N= 38] =.11, 
p = .744), age of participating parent (¢[35] = 1.84, 
p = .074), or the number of children in the household 
(¢(36] = .81, p= .421). All intervention and control 
teachers were female. Most teachers in the intervention and 
control conditions were either Caucasian (79% vs. 63%; va 
{1, N = 38] = 1.15, p = .283) or African-American (11% 
vs. 21%; y° [1, N = 38] = .79, p = .374). The percent of 
teachers who self-reported attaining a bachelor’s degree or 
higher was equivalent across the two conditions (58% for 
both conditions). Teachers in control classrooms reported 
teaching for more years (M[SD] = 18.6[8.2]) than teachers 
in intervention classrooms (M[SD] = 12.0[8.8]; 
t(36] = 2.40, p = .022). 


Attrition and Missing Data 


For the 38 children examined in this report, baseline and 
posttest questionnaire data were available for 37 of 38 
(97%) teachers and 35 of 38 (92%) parents. Posttest data 
were not available for one teacher and three parents. 
Although we obtained parent-reported and teacher-reported 
data at both time points for most informants, missing item- 
level data precluded scale scoring on occasion. Two 
additional teachers were missing two of six teacher-re- 
ported outcomes and two additional parents were missing 
one of the three parent-reported outcomes due to missing 
item-level data. We examined the assumption that data 
were missing completely at random using Little’s MCAR 
test. The test was nonsignificant (y* = 70.58, p = .492) 
suggesting data were missing completely at random. 


Fidelity, Program Compliance, Alliance, 
and Satisfaction 


Descriptive statistics for child, parent, and teacher process 
measures are summarized in Table 3. Coaches and teachers 
adhered to over 95% of core program components. For 
coaches, quality of implementation of the CLASS com- 
ponent was excellent (.94; range .83-1.00), whereas for 
teachers, it was acceptable (.80; range .41—1.00). Children 
in the comorbid sample received 93% (range 50-100%) of 
program days and families received 86% (range 17—100%) 
of homeBase sessions. Although child compliance was 
excellent (.90) with respect to the CLASS component, 
parent compliance for the homeBase component was low 
(.56). Coach-reported parent fidelity was also low (.61). 
Coach-reported and teacher-reported alliance was high 
based on mean ratings, and both informants reported high 
levels of satisfaction with the program. 


Table 2 Student baseline demographic characteristics and screening measures by study condition 


Total (n = 38) Control (n = 19) Intervention (n = 19) Test statistic p value 

Demographic characteristic 

Age M (SD) 4.1 (.2) 4.1 (.2) 4.1 (.2) 00 1.00 

Percent female 10 (26.3) 5 (26.3) 5 (26.3) 00 1.00 

Percent African-American 11 (28.9) 3 (15.8) 8 (42.1) 3.20 074 

Percent Caucasian 19 (50.0) 11 (57.9) 8 (42.1) 95 330 

Percent on IFSP 11 (29.7) 5 (27.8) 6 (31.6) .06 .800 
Screening measures 

Percent ranked Ist 33 (86.8) 15 (78.9) 18 (94.7) 2.07 150 

Aggressive behavior scale M (SD) 22.3 (6.2) 22.4 (6.7) 22.2 (6.0) 10 919 

Adaptive behavior index M (SD) 22.1 (3.8) 22.6 (3.4) 21.5 (4.2) 94 354 

Maladaptive behavior index M (SD) 31.0 (5.7) 31.4 (4.7) 30.6 (6.6) 40 694 


Reported test statistics are ¢ for continuous and Chi-square for dichotomous measures 


IFSP individual family service plan 
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Table 3 Descriptive statistics 


ar antadveution process Classroom M (SD) HomeBase M (SD) Overall 

measures Coach Teacher Combined Parent 
Protocol adherence .95 (.07) .96 (.08) 95 (.07) - .95 (.07) 
Quality of implementation .94 (.05) .80 (.16) .87 (.08) 61 (.36) 80 (.14) 
Dosage - - 93 (.14) .86 (.23) 89 (.14) 
Compliance - - 90 (.13) 56 (.44) -79 (.23) 
Alliance 4.45 (.68) 4.88 (.27) —- - 4.67 (.43) 
Satisfaction - 4.32 (.63) - 4.51 (.57) 4.39 (.47) 


Posttest Differences on Outcome Measures 


Table 4 summarizes results from  covariate-adjusted 
regression models. Children randomized to the First Step 
intervention exhibited significant reductions in externaliz- 
ing behavior based on teacher and parent report on all four 
outcome measures and statistically significant improve- 
ment in social functioning on the three teacher- and parent- 
reported outcome measures. For reductions in externalizing 
behavior, Hedges’ g effect sizes for teacher-reported 
measures ranged from .59 to .79. For improvements in 
functioning, effect sizes for teacher-reported measures 
ranged from .60 to .85. For parent-reported outcomes, 
Hedges’ g effect sizes ranged from .66 for reduction in 
externalizing behavior to .61 for improvements in social 
functioning. Reductions in internalizing behavior were not 
statistically significant for the two conditions with effect 
sizes in the small range for parent-reported (.23) and small- 
to-medium range for teacher-reported (.42) internalizing 
behavior. 


Practical Significance of Intervention Effects 


For externalizing behavior, mean improvement across the 
four outcomes was +26 percentile points (SSiS-EXT- 
T= + 22; SSiS-EXT-P = + 25; ESP-MBI-T = + 26; 
ESP-ABS-T = + 29). For social functioning, mean 
improvement across the three outcomes was +25 per- 
centile points (SSiS-SS-T = + 23; SSiS-SS-p = + 23; 
ESP-ABI-T = + 30). For internalizing behavior, the 
improvement index ranged from +9 percentile point 
improvement based on parent report to +16 percentile 
point improvement based on teacher report. 


Discussion 


This report advances the literature regarding the differen- 
tial impact of the First Step intervention on preschoolers at 
risk of heterotypic comorbid psychiatric disorders such as 
disruptive behavior and anxiety disorders. Such findings 


Table 4 Baseline and posttest means and standard deviations for outcome measures by condition and regression results 


Intervention (n = 19) 


Condition effect Effect size 


Domain/measure Control (n = 19) 
Baseline Posttest Baseline 
M (SD) M (SD) Mag  M (SD) 
Internalizing behavior 
SSiS-INT-T 6.3 (3.5) 5.0 (2.5) 4.8 5.3 (3.7) 
SSiS-INT-P 10.2 (5.6) 9.6 (5.4) 8.7 7.6 (3.9) 
Externalizing behavior 
SSiS-EXT-T 19.4 (7.4) 16.4 (6.4) 15.7 17.6 (4.9) 
SSiS-EXT-P 19.9 (8.8) 18.1 (8.3) 18.2 19.7 (5.8) 
ESP-MBI-T 29.1 (6.6) 26.0 (7.2) 26.2 28.2 (7.0) 
ESP-ABS-T 21.6 (7.8) 20.3 (10.2) 19.4 18.9 (5.6) 
Social functioning 
ESP-ABI-T 23.9 (4.3) 26.8 (5.9) 26.3 24.2 (4.4) 
SSiS-SS-T 77.6 (12.4) 86.6 (16.0) 86.4 771A (8.7) 
SSiS-SS-P 90.4 (11.1) 93.6 (13.1) 94.2 88.6 (11.4) 


Posttest 
M (SD) Magj Test statistic p value Hedges’ g 
3.5 (2.7) 3.) —1.69 091 42 
6.6 (4.7) 75 —.91 363 .23 
11.8 (5.7) 12.2 —2.33 .020 59 
13.5 (4.8) 13.8 —3.07 .002 .66 
21.3 (6.2) 21.5 —2.55 O11 -70 
12.9 (3.1) 13.5 —2.50 012 719 
31.1 (5.2) 31.0 2.82 005 85 
95.4 (13.5) 95.4 2.51 012 60 
102.3 (14.5) 102.6 2.45 014 61 
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are important given not only that anxiety disorders have an 
estimated prevalence rate as high as 9 percent among 
preschool children (Bufferd, Dougherty, Carlson, Rose, & 
Klein, 2012), but also that the high comorbidity of anxiety 
disorders with disruptive behavior disorders (Franz et al., 
2013) portends poorer treatment outcomes for children who 
present with multiple symptoms of both disorders (Garcia 
et al., 2010; Hudson et al., 2015; Rapee et al., 2013). In this 
subsample analysis, baseline equivalence between the First 
Step and control conditions was established, and the 
implementation measures for the intervention such as 
fidelity and satisfaction were comparable to those obtained 
in the original trial involving the total sample (Feil et al., 
2014). 

The magnitude of the intervention effects on external- 
izing behavior and social functioning measures were 
somewhat comparable to those obtained for the original 
trial of preschoolers with effect sizes ranging from medium 
to large. For example, on the four teacher-reported 
behavioral and social outcome measures (i.e., the three ESP 
measures and the SSiS social skills measures) that were the 
same as those used in the original trial (Feil et al., 2014), 
the average effect size for the current study was .74 (range 
.60—.85) compared to .77 (range .70-.88) in the original 
trial. Hence, there appears to be only a slight decrease in 
the efficacy of First Step on disruptive behavior and social 
skills due to the presence of comorbid anxiety in this 
subsample of preschoolers contrary to the previous 
research on the generally negative impact of this comor- 
bidity on treatment outcomes cited in the introduction. 

With respect to the impact of First Step on the inter- 
nalizing behavior measures on the other hand, the effects 
were more equivocal with small-to-medium effects 
obtained for parent report (.23) and teacher report (.42). 
Hence, although the First Step intervention appears to have 
a relatively commensurate impact on externalizing behav- 
ior problems and social skills for preschoolers with 
comorbid anxiety symptoms, the broadband behavioral 
intervention had a positive but nonetheless limited impact 
on internalizing behaviors. This is somewhat surprising 
given that some components of the First Step program 
might have been expected to produce clinically meaningful 
improvement in children’s internalizing symptoms. Such 
components may have contributed to gains on disruptive 
behavior in the comorbid anxiety subsample but did not 
appear sufficient to meaningfully impact their internalizing 
symptoms. There is evidence that cognitive—behavioral 
treatment of anxiety disorders per se reduces comorbid 
disruptive behaviors by as much as a third (Rapee et al., 
2013), but whether the reverse is true (i.e., treatment of 
disruptive behavior can significantly reduce anxiety disor- 
ders) is not borne out by the secondary analyses conducted 
for this report. 


The tentative implications of this ancillary study seem 
relatively straightforward. On the one hand, the presence of 
comorbid anxiety symptoms do not appear to substantively 
impact the behavioral or social functioning outcomes of a 
generic behavioral intervention such as First Step, although 
it remains important to replicate these findings with larger 
samples. On the other hand, the First Step intervention had 
a limited impact on internalizing behaviors, particularly as 
reported by parents. This latter finding is not surprising 
given that these preschoolers were originally selected by 
teachers for their externalizing problem behaviors and the 
First Step program was primarily designed to reduce dis- 
ruptive and aggressive behavior. It is worth noting that 
early signs of internalizing behavior seem notoriously 
difficult for teachers to recognize let alone address (Bri- 
esch, Ferguson, Volpe, & Briesch, 2013; Burke et al., 
2012; Gilliam, Maupin, & Reyes, 2016). This is in spite of 
the fact that there is increasing evidence that anxiety dis- 
orders can adversely impact academic performance (Killu 
& Crundwell, 2016; Schoenfeld & Janney, 2008). 

Although this subsample analysis has some considerable 
strengths given the carefully controlled efficacy trial from 
which it was drawn, there are several limitations that 
should be noted. The subsample of preschoolers was small, 
which limits the statistical power for the study. Given the 
small sample, we could only examine an overall at-risk 
anxiety disorder group rather than differentiating between 
the various types of anxiety disorders such as fear-based 
disorders (e.g., separation anxiety, phobias, OCD) and 
distress-based disorders (e.g., GAD, PTSD). In addition, 
parent symptom checklists were used to identify 
preschoolers at risk of comorbid anxiety disorders rather 
than using the gold standard of a child diagnostic interview 
schedule. Given these limitations, this subsample analysis 
serves primarily as a hypothesis-generating approach and 
the practice-based implications must await further 
confirmation. 

With the widespread adoption of multitiered systems, 
based on the public health model of universal, selected, and 
indicated prevention within school settings, early evidence- 
based approaches such as the First Step program hold great 
promise for reducing the incidence of behavioral disorders 
and the associated negative sequelae such as poor social 
and academic functioning (Walker, et al., 1996). Though 
the evidence base for the effectiveness of generic broad- 
band behavioral interventions with multiple tiers for 
addressing externalizing behavior problems is mounting 
(Walker & Gresham, 2014), more limited evidence exists 
to date for the early prevention of internalizing disorders 
(Fisak, Richard, & Mann, 2011). Furthermore, although the 
design of similar broadband behavioral or transdiagnostic 
intervention approaches to address comorbid conditions are 
in a nascent stage at this time, such approaches seem to 
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compare relatively well in effectiveness to disorder-speci- 
fic interventions (McEvoy, Nathan, & Norton, 2009; 
McHugh, Murrary, & Barlow, 2009; Weersing et al., 
2017). In this regard, we should note similar subsample 
analyses from our original randomized controlled trial: one 
on preschoolers at risk of ADHD (Feil et al., 2016) and the 
other on preschoolers at risk of autism spectrum disorder 
(Frey et al., 2015). Both not only produced large positive 
effects on disruptive behavior and social functioning but, in 
contrast to our current findings, also produced medium-to- 
large effects on comorbid ADHD and autism symptoms, 
respectively. Should these effects, along with the current 
findings, be replicated in larger a priori studies rather than 
post hoc subsample analyses, it would suggest a clear need 
to coordinate both early identification and intervention 
efforts to more effectively monitor the specific mental 
health needs of children at risk of comorbid 
psychopathology. 
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